Method of analyzing conversational transcripts

ABSTRACT

Analyzing transcripts of conversation between at least two users by receiving input information from a first user via a voice call, creating conversational transcripts from the information received from the first user, selecting at least one defined situation from a list of defined situations, identifying the selected situation in the conversational transcripts, identifying a set of procedural sequences by comparing the at least one identified situation in the conversational transcripts with knowledge derived from a corpus of historical conversational transcripts; and providing the set of procedural sequences to the first user.

FIELD OF THE INVENTION

This invention relates to a method of analyzing conversationaltranscripts, preferably textual conversational transcripts, and inparticular to analyzing and processing of unstructured data in suchconversational transcripts.

DESCRIPTION OF THE RELATED ART

Call centers are centralized offices for the purpose of receiving andtransmitting a large volume of requests by various communicationchannels like telephone, emails, and instant messages. A call centre isoperated by a company to administer various types of support orinformation inquiries from customers. This call center model is becomingpopular because it allows companies to be in direct touch with theircustomers. In such call centers a typical call center agent handles tensto hundreds of calls per day depending on the complexity of the issuesthat the agent is required to addresses. However, with advances in thearea of speech recognition and the widespread deployment of speechrecognition technology in call centers, relatively large volumes of dataare produced everyday in the form of call transcripts. This is inaddition to the data produced as a part of the process which includee-mail, instant messages (IM), reports from the agent, customersatisfaction surveys etc. In the sequel, the conversation (call)transcripts includes transcripts of conversations through various medialike audio calls and IM/web-chat.

A typical call that an agent would receive and respond to at a callcenter consists of various phases. For example, a call to a call centreagent broadly consists of the following three phases: introduction partbetween the agent and the customer, seeking information about theproblem, and providing solution to the problem of the customer. Agentsin call centers typically use keyword search over manually authoredknowledge bases to obtain useful information that could help them tosolve the problem faced by the customer. The utility of such a system islimited by the smartness of the agent to enter the right keywords.

Knowledge authoring, or the act of populating the knowledge base is donemanually. Because they are manually authored, these knowledge bases cannot quickly adapt to the various kinds of new problems that arise duringthe course of time. Moreover, manual augmentation of such knowledgebases would consume tremendous manpower and may not be exhaustive.Identification of undocumented and effective sequences of steps would beimpossible in the case of manual authoring of knowledge bases.

Without a way to improve the method of analyzing call transcripts, thepromise of this technology may never be fully achieved.

SUMMARY OF THE INVENTION

According to a first aspect of the invention thereof, is described amethod for analyzing transcripts of conversation between at least twousers. Typically the call center agent is configured to receive inputinformation from first user (customer, etc.) via a voice call. Afterreceiving the call, conversational transcripts are created from theinformation received from the first user. Once the conversationaltranscripts have been created at least one defined situation from a listof defined situations is selected. The selected situation is identifiedin the conversational transcripts. After the situations has beenidentified, a set of procedural sequences is identified by comparing theat least one identified situation in the conversational transcript withknowledge derived from a corpus of historical conversationaltranscripts, which are preferably available in a corpus. The set ofprocedural sequence is provided to the first user. In a furtherembodiment, the set of procedural sequence may be provided to the seconduser (call center agent) and the second user can provide the proceduralsequence to the first user.

According to a second aspect of the invention thereof, is a method forcreating a document of procedural sequences from information receivedvia an input call. Typically the call center agent is configured toreceive input information from first user (customer, etc.) via a voicecall. After receiving the call, conversational transcripts are createdfrom the information received from the first user. Once theconversational transcripts have been created, at least one definedsituation from a list of defined situations is selected. The selectedsituation is identified in the conversational transcripts. After thesituations have been identified, a set of procedural sequences isidentified by comparing the at least one identified situation in theconversational transcript with knowledge derived from a corpus ofhistorical conversational transcripts, which are preferably available ina corpus. On negative determination, i.e., when the identified situationis not found in the historical conversational transcripts, the seconduser (call center agent) is prompted to create a document comprising aset of procedural sequences and add the set of procedural sequences intoa repository. The document created is authored by the second user andsuch a document that is added into the repository can be advantageouslyused in identifying situations and providing procedural sequences to theidentified sequences from the input call of the first user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates an exemplary embodiment of a method of analyzing calltranscripts in accordance with this invention.

FIG. 2 illustrates an exemplary embodiment of analyzing conversationaltranscripts and generating documents.

FIG. 3 illustrates an exemplary embodiment an analyzing conversationaltranscripts.

FIG. 4 illustrates an exemplary embodiment of call processing using aclustering algorithm.

FIG. 5 illustrates an exemplary embodiment an algorithm used in sequencemining on call corpora.

FIG. 6 illustrates an exemplary embodiment of authoring new knowledgeand updating the knowledge base.

DETAILED DESCRIPTION Overview

Collections and analysis of call transcripts are typically very diversein the kind of predicament they address. The call transcripts areclustered initially to arrive at topical collections, which arecollections of calls addressing a specific issue, where such topicalcollections are more homogeneous in nature and can be advantageouslyused for analysis of the call transcripts. Each such topical collectionof calls is further split into two subsets, that of second user(hereinafter referred to as call center agent or simply agent) sentencesand first user (also referred to as a customer) sentences. The twosubsets are further clustered to build clusters which possiblycontaining sentences representative of similar procedural steps, whereeach cluster thus describes a sub-procedural step. An entropy-basedmeasure to quantify the quality of a sub-procedural cluster is defined,and the entropy is used to measure the cluster quality and to refineclusters until good quality clusters are found. Once these clusters areobtained, each call in the topical collection can now be represented asa sequence of such clusters. The collection of calls, represented assequences, are subjected to frequent pattern mining to discover frequentprocedural sequences for a topical collection. Representativedistinctive procedural sequences are determined from these frequentsequences, using for example, techniques such as leader clusteringalgorithms.

Method of Analysing Conversational Transcripts

FIG. 1 illustrates an exemplary embodiment of a method 100 which is theintegrated approach of analyzing call transcripts. In 120, the methodinvolves performing clustering of the corpus of call transcripts inorder to group related text segments. Each group of segments discoveredin 120 is represented by a representative syntax in 130. These groups ofsegments are also referred to as the sub-procedure text segments (STPS).In 140, the calls are represented as a sequence of STPS. Once the callshave been represented as a sequence of STPS, in 150 mining is performedon the sequence to determine the frequent sequences across the calls.After this, in 160 a clustering algorithm such as CAARD is used to getdistinct and long sequences from the sequences that are mined and theresults are used in subsequent embodiments.

Reference is now made to FIG. 4 which illustrates, in more details, anexemplary embodiment of a method 400 which is the integrated approach ofanalyzing call transcripts. In 410, the method involves performingclustering of the corpus of call transcripts in order to findtopic-specific clusters S_(i) of calls received. In 420, each clusterS_(i) collects the set of agent (second users) sentences A_(i) and theCustomer sentences (first users) C_(i). In 430, the sentences A_(i) andC_(i) p clustered to find the sub-procedure text segments (STPS). In 440the entropy which defines the quality of the STPS is computed and in 445a check is made on the entropy value, which gives the quality of theSTPS. If the entropy value computed is not good, i.e., does not satisfya predefined criteria, in 450 the clustering parameters are changed andthe 430 is repeated until the entropy value of the STPS computed isgood, i.e., above a predefined threshold. In 445, if the entropy valueis determined to be good, i.e., above a predefined threshold then, in460, the calls are represented as a sequence of cluster labels. Once thecalls have been represented as a sequence of cluster labels, in 470mining is performed on the sequence to determine the frequent sequencesacross the calls S_(i). After this, in 480 a clustering algorithm suchas CAARD is used to get distinct sequences from the sequences that aremined and the results are output to the first user. The output from thealgorithm may be provided directly to the customer or via the agent tothe customer.

While clustering the textual segments in the corpus of the historicalconversational transcripts, the historical transcripts are organizedinto groups and conversational transcripts in accordance with predefinedcriteria, and the groups of conversational transcripts to obtain groupsof related textual segments. While clustering groups of conversationaltranscripts to obtain groups of related documents, at least two set ofdocuments from the groups of documents corresponding each belonging tothe customer and the agent are generated. Typically, a K-meansclustering is performed on the corpus to obtain groups of conversationson a single topic, to extract coherent and meaningful SPTS from theconversation transcripts. For each such topical cluster, the set ofagent sentences and customer sentences are collected separately, andthey are clustered adaptively in accordance with 145, until good qualitySPTS are obtained.

In a further embodiment, in order to make the output more readable, eachSPTS cluster is represented by sentences that characterize the cluster.These characteristic sentences can be obtained using one of textsummarization algorithms. The descriptive and discriminative words fromthe sentences in the cluster are used to describe the cluster.Descriptive words are those that contribute most to the averagesimilarity among the calls in the cluster. Discriminative words arethose that are more prevalent in the calls in the cluster as compared tothe rest of the calls.

To Provide Information to the Call Participants

FIG. 2 illustrates an exemplary embodiment of a method 200 of analyzingcall transcripts in accordance with the claimed invention. In 210, aninput call is received from a customer by an agent. On receiving thecall, for example an input voice call, call via an instant messenger,call via email or other means known to a person skilled in the art,preferably a voice call, in 220 conversational transcripts are createdfrom the information received from the call of the customer. Aftercreating conversational transcripts, in 230 at least one definedsituation is selected from a list of defined situations, where eachsituations is uniquely defined by a set of parameters. In 240, from theset of defined situations that are derived from the conversationaltranscript, at least one situation is selected. This selection can bebased on a threshold value which has been predefined. Once a selectedsituations in the conversational transcripts has been identified in 240,a set of procedural sequences is identified by comparing the at leastone identified situation in the conversational transcript with knowledgederived from a corpus in 100, for example a set of distinct and longfrequently occurring subsequence of STPS as in 160. Once the set ofprocedural sequences has been identified, in 170, the customer isprovided with the identified procedural sequences, either directly orvia the agent.

To Author New Information to Include in Knowledge-Base

FIG. 3 illustrates an exemplary embodiment of a method 300 of analyzingthe call transcripts in order to determine and author new information toincorporate in an already existing knowledge-base. In 320,conversational transcript of the received call is created. From theconversational transcript, at least one defined situation in the list ofdefined situations is identified in 330. In 340, the identifiedsituation is compared with the knowledge repository derived from thehistorical conversation as in 100. Once identified situation is notfound in knowledge repository, a new document comprising a set ofprocedural sequences is created and added to the knowledge repository.

Determining the Quality of the Cluster Call Entropy: A Measure ofScatter for a Call

A typical call consists of a sequence of information exchanges andassumes that there exist some sentences in a conversation that reflectsub-procedures steps. The goodness measure (hereinafter referred to asgoodness) of the SPTS cluster can be partially judged by the frequencyof calls that contains sentences from each SPTS cluster, and the numberof calls the sentences in each SPTS cluster are scattered into. Thisintuition is quantitatively captured by defining entropy of a call withrespect to the SPTS clusters. Let, Call1 be represented by the sequence<C₂, C₁, C₅, C₆, C₄> and Call2 by the sequence <C₃, C₅, C₅, C₃, C₅>. Asis obvious from the representation, Call1 is more scattered than Call2.The entropy measure is used to measure the scatter of a call. Let,E_(C)(c) represent entropy with respect to a given set C of SPTSclusters. The entropy measure is computed using Eq. (1):

E _(C)(c)=Σ_(i) d _(i) log(d _(i)),  (1)

where in Eq. (1), d_(i) is the fraction of the call C in a cluster i.For example, for the clusters numbered C₁ to C₆, E(Call1)=0.6989, andE(Call2)=0.29.

Normalized Entropy: Adaptation to Compare Calls Across Cardinalities

Entropy is satisfactory measure to compare calls of the samecardinality. Consider the case of Call3 which is <C₁, C₂> and Call4which is <C₁, C₁, C₁, C₁, C₂, C₂, C₂, C₂>, which have the same entropy.Therefore, Call3 should have a higher score since it is scattered acrossas many clusters as it can be. Thus, a Normalized Entropy (NE) isproposed which is a better measure of scatter of the call. NE of a callc with respect to a set C of clusters is defined as

NE _(C)(c)=−(Σ_(i) d _(i) log(d _(i)))/log(|c|),  (2)

where in Eq (2), |c| is the length of the call c. Normalized Entropywould lie between a value between 0 and 1 since log(|c|) is the maximumvalue that Normalized Entropy takes. Call3 would have an NE value of 1.0and Call4 would have an NE value of 0.333.

Quality Measure for a Clustering on a Collection of Calls

Thus, a quality measure for a given set C of clusters is defined usingthe normalized entropy of all the calls in the corpus R, as thecardinality weighted average of NE values for calls as defined in Eq.(3):

NE _(R)(C)=(Σ_(c ε R) NE _(C)(c)*|c|)/(Σ_(c ε R) |c|).  (3)

Important properties of the NE measure:

-   -   NE increases with the number of clusters because, there are more        clusters for a given call to get scattered into.    -   For a given number of clusters and an approximately equal number        of data elements, NE decreases with the increase in average call        length. This is due to the increased probability of two steps        (in the call) getting mapped to the same cluster with increase        in the length of a call.        NE is parameterized by both the clusters and the corpus. Hence,        a comparison can be made between the clustering algorithms by        their NE values on various corpora, and different corpora by        their NE values when subjected to the same clustering technique.        However, the characteristics of the NE function outlined        previously leads to two important requirements for NE value        based comparison. The clusters to be compared should have        roughly same values for the following ratios    -   Sentences ratio: Number of sentences to be clustered/Number of        clusters    -   Call length ratio: Average Call Length/Number of clusters

Analysis of Corpus Homogeneity Using NE Measure

A corpus of call collections would contain calls pertaining to diverseissues. A clustering of agent sentences and customer sentences of such acorpus, would lead to topic-type clusters than SPTS clusters. Ahomogeneous corpus could be expected to give better SPTS clusters than amore diverse collection, based on the assumption that the largerdissimilarities (which are topic-level) have to be eliminated in orderto expose the lesser (SPTS) dissimilarities to the clustering algorithm.Therefore, in this invention it is hypothesized that NE of the clustersincreases with the homogeneity of the corpus of calls to be clustered.

Mining of STPS Cluster Sequences Frequent Sequences

This describes the technique of extracting useful information from thecall sequences by means of frequent sequence mining. Mining sequentialpatterns from data is well known to a person skilled in the art.Algorithms available in the art can be used to find frequent sequencesfrom the set of call sequences. However, these algorithms result in manyredundant sequences. Hence, summarization techniques like CAARD to findlong and distinct sequences can be advantageously used. Such a sequencemining approach 300 has been illustrated in FIG. 3 where in accordancewith this invention distinct frequent sequences can be extracted.

Calls as Cluster Sequences

Let, {C₁, . . . , C_(n)} be the clusters of sentences corresponding tocustomer and {A₁, . . . , A_(m)} be that to agents. Each call isrepresented by a sequence of those clusters to which sentences in thecall belong to. For example, <START, A₂, C₁, A₄, C₄, END> represent acall where the first sentence in the call, which is in A₂, was spoken byagent, second sentence, which is in C₁, is spoken by customer, and soon. Without any loss of generality it can be assumed that the callconsists of sentences belonging to As and Cs alternatively asillustrated in FIG. 4. FIG. 4 illustrates an exemplary embodiment 400 ofsplitting calls received as inputs into agent sentences and customersentences, clustering these sentences and representing calls as asequence of clusters. Consecutive sentences by the same person can becombined into a single sentence. Once the distinct sequences of SPTSclusters, these sequences are added to the procedure repository

EXAMPLES Agent Prompting Application using SPTS Cluster SequencesRepository

Consider, an agent prompting application is an online process asillustrated in FIG. 5, consisting of extraction of procedures from avoice call. An ASR (Automatic Speech Recognition) system at any timeduring the call has the transcript of the call up to that point in time.A call transcript to the procedure matcher happens periodically in thecall (say, once in 30 seconds or so), where the procedure matcher findsthe closest matching procedure(s) from the repository of procedures, asdescribed by the method in accordance with this invention previously, tothe call transcript. The matched procedure(s) is (are) displayed to theagent. The agent would be able to take clues from the displayed matchingprocedure(s) to decide on actions to be taken in the remaining part ofthe call.

Assume that a laptop help-desk agent receives a call on resetting a userpassword. There could be two ways of resetting the user password inWindows OS:

-   -   1. pressing Ctrl+Alt+Delete and press on change password button        to change the password, and    -   2. opening control panel, clicking on “user accounts” icon to        change the password.

These two procedures if present in the procedure database are retrievedand shown to the agent while taking the call on user password resettingin Windows OS Knowledge Authoring Application using SPTS ClusterSequences Repository

The Knowledge Authoring application is an offline process aimed atcapturing any new knowledge which has not been recorded in the procedureDB for possible inclusion into the knowledge base. FIG. 3 illustrates anexemplary embodiment of use of extracted procedures in knowledgeauthoring. This is a semi-supervised process where whether or not thesuggested new knowledge is worthy of inclusion into the knowledge baseis left to the discretion of the knowledge author.

For example, suppose a laptop helpdesk agent, while conversing with acustomer, has diagnosed and fixed an unknown problem or diagnosed aknown problem but fixed the problem in a way that is not captured in theknowledge base that he typically uses. If this type of the call isdiscovered then, the person who is responsible for maintaining theknowledge base, could generate a document related to this call,capturing the new knowledge and store the new knowledge in the knowledgebase. After completion of a call, the call transcript is fed into theprocedure matcher that finds procedure that match with the call (basedon a threshold). If matching procedure for the call is not found, theknowledge author is prompted to possibly include the new knowledgecaptured into a document format and then updated into the knowledgebase.

Although the invention has been described with reference to theembodiments described above, it will be evident that other embodimentsmay be alternatively used to achieve the same object. The scope of theinvention is not limited to the embodiments described above, but canalso be applied to software programs and computer program products ingeneral. It should be noted that the above-mentioned embodimentsillustrate rather than limit the invention and that those skilled in theart will be able to design alternative embodiments without departingfrom the scope of the appended claims. In the claims, any referencesigns should not limit the scope of the claim. The invention can beimplemented by means of hardware and software comprising severaldistinct elements.

1-11. (canceled)
 12. A method for creating a document of proceduralsequences, the method comprising: receiving input information from afirst user; creating conversational transcripts from the informationreceived from the first user; identifying at least one defined situationin the list of defined situations from the conversational transcripts;comparing the situations with knowledge derived from historicalconversational transcripts in a corpus; on negative determinationcreating a document comprising a set of procedural sequences; and addingthe document created to a repository; and prompting a second user oncreating a set of procedural sequences based on the defined situationwherein knowledge derived from a corpus of historical conversationaltranscripts comprises: clustering textual segments in the corpus ofhistorical conversational transcripts into groups of related textualsegments: representing each of the groups of related textual segments bya representative syntax: forming a sequence of representative syntax foreach historical conversational transcript in the corpus: findingfrequently occurring sub sequences from the sequences of representativesyntax; and finding distinct and long frequently occurring sub sequencesfrom the frequently occurring sub sequences, wherein clustering thetextual segments in the corpus of historical conversational transcriptsfurther comprises: organizing the historical conversational transcriptsinto groups of conversational transcripts according to a definedcriteria; and clustering the groups of conversational transcripts toobtain groups of related textual segment, wherein clustering groups ofconversational transcripts to obtain groups of related documents furthercomprises: generating at least two sets of documents from the groups ofconversational transcripts, wherein each of the sets of documentscorresponds to at least one of the users, wherein each of therepresentative syntax comprises a segment of conversational transcriptsbelonging to each of the group of related textual segments, wherein eachof the representative syntax comprises a set of frequent and distinctwords in each of the group of related textual segments, wherein findingdistinct and long frequently occurring sub sequences is performed by aclustering algorithm belonging to the class of leader clusteringalgorithm, wherein the corpus of historical transcripts comprises arepository, and wherein clustering textual segments in the corpus ofhistorical conversational transcripts into groups of related textualsegments comprises utilizing a set of configurable cluster parameters tocluster the textual segments based on normalized entropy values. 13-21.(canceled)